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The Incredible Years 

No studies of The Incredible Years that fall within the scope of the Early Childhood Education Interven- 
tions for Children with Disabilities review protocol meet What Works Clearinghouse (WWC) evidence 
standards. The lack of studies meeting WWC evidence standards means that, at this time, the WWC is 
unable to draw any conclusions based on research about the effectiveness or ineffectiveness of The 
Incredible Years on preschool children with disabilities in early education settings. Additional research 
is needed to determine the effectiveness or ineffectiveness of this intervention. 

Program Description 1 

The Incredible Years is an intervention composed of training programs for children, parents, and teachers that is 
intended to reduce children’s aggression and improve their social skills. The three programs (child, parent, and 
teacher) can be used independently or in combination. 

The child program focuses on building social and emotional skills for students in preschool through early elemen- 
tary school. The child program can be delivered to children referred for difficult behavior or to an entire classroom 
as a preventive measure. This program can be offered in either a classroom or a clinical setting. In a classroom 
setting, the teacher presents 20- to 30-minute lessons two to three times a week during circle time. Alternately, the 
child program can be conducted as a pullout program for small groups of children. Lessons cover recognizing and 
understanding feelings, getting along with friends, anger management, problem solving, and behavior at school. In 
a clinical setting, referred students complete activities during 18-20 two-hour weekly small-group meetings. 

Parent training programs focus on parenting skills, including positive discipline, promoting child learning and devel- 
opment, and involvement in children’s life at school. The program also emphasizes parent interpersonal skills to 
improve communication with children and other parents. 

The Incredible Years also includes two programs for teachers. The first addresses general classroom manage- 
ment as a means to improve student behavior and learning. The second is a training program for teachers who will 
deliver the child program to students in their classrooms. 

Research 2 

The WWC identified 166 studies of The Incredible Years for preschool children with disabilities in early education 
settings that were published or released between 1 989 and 201 1 . 

Three studies are within the scope of the Early Childhood Education Interventions for Children with Disabilities 
review protocol but do not meet WWC evidence standards. 

• One study was a randomized controlled trial in which the combination of overall and differential attrition 
exceeded WWC evidence standards. The resulting analytic intervention groups were too dissimilar at baseline to 
be considered equivalent. Therefore, this study did not meet WWC evidence standards. 

• Two studies have the delivery of the intervention as a confounding factor, which makes it impossible to attribute 
the observed effect solely to The Incredible Years. In one study, the author led all of the sessions for the interven- 
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tion group and did not lead the control group sessions. In the second study, the intervention sessions were led 
by the same two-person team, while the control group sessions were led by different individuals. In both of these 
studies, reported program effects may be attributed to the individual(s) who delivered The Incredible Years rather 
than to the intervention itself. 

Seventy-two studies are out of the scope of the Early Childhood Education Interventions for Children with Disabilities 
review protocol because they have an ineligible study design. 

Ninety-one studies are out of the scope of the Early Childhood Education Interventions for Children with Disabilities 
review protocol for reasons other than study design. 
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Endnotes 

1 The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www. 
incredibleyears.com/, downloaded February 201 1). The WWC requests developers to review the program description sections for 
accuracy from their perspective. The program description was provided to the developer in June 201 1 and we incorporated feedback 
from the developer. Further verification of the accuracy of the descriptive information for this program is beyond the scope of this 
review. The literature search reflects documents publicly available by August 201 1 . 

2 The studies in this report were reviewed using WWC Evidence Standards, Version 2.1, as described in protocol Version 2.0. The 
evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes 
available. 
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Glossary of Terms 

Attrition 



Clustering adjustment 
Confounding factor 

Design 
Domain 
Effect size 

Eligibility 

Equivalence 

Extent of evidence 

Improvement index 

Multiple comparison 
adjustment 

Quasi-experimental 
design (QED) 

Randomized controlled 
trial (RCT) 

Rating of effectiveness 



Single-case design 
Standard deviation 



Statistical significance 



Substantively important 



Attrition occurs when an outcome variable is not available for all participants initially 
assigned to the intervention and comparison groups. The WWC considers the total attrition 
rate and the difference in attrition rates across groups within a study. 

If treatment assignment is made at a cluster level and the analysis is conducted at the student 
level, the WWC will adjust the statistical significance to account for this mismatch, if necessary. 

A confounding factor is a component of a study that is completely aligned with one of the 
study conditions, making it impossible to separate how much of the observed effect was 
due to the intervention and how much was due to the factor. 

The design of a study is the method by which intervention and comparison groups were assigned. 
A domain is a group of closely related outcomes. 

The effect size is a measure of the magnitude of an effect. The WWC uses a standardized 
measure to facilitate comparisons between outcomes and studies. 

A study is eligible for review and inclusion in this report if it falls within the scope of the 
review protocol and uses either an experimental or matched comparison group design. 

A demonstration that the analysis sample groups are similar on observed characteristics 
defined in the review area protocol. 

An indication of how much evidence supports the findings. The criteria for the extent of 
evidence levels are given in the WWC Procedures and Standards Handbook (version 2.1). 

Along a percentile distribution of students, the improvement index represents the gain or 
loss of the average student due to the intervention. As the average student starts at the 
50th percentile, the measure ranges from -50 to +50. 

When a study includes multiple outcomes or comparison groups, the WWC will adjust the 
statistical significance to account for the multiple comparisons, if necessary. 

A quasi-experimental design (QED) is a research design in which subjects are assigned to 
treatment and comparison groups through a process that is not random. 

A randomized controlled trial (RCT) is an experiment in which investigators randomly assign 
eligible participants into treatment and comparison groups. 

The WWC rates the effects of an intervention in each domain based on the quality of the research 
design and the magnitude, statistical significance, and consistency in findings. The criteria for the 
ratings of effectiveness are given in the WWC Procedures and Standards Handbook (version 2.1). 

A research approach in which an outcome variable is measured repeatedly within and 
across different conditions that are defined by the presence or absence of an intervention. 

The standard deviation of a measure shows how much variation exists from the average. A 
low standard deviation indicates that the data points tend to be very close to the mean; a high 
standard deviation indicates that the data points are spread out over a large range of values. 

Statistical significance is the probability that the difference between groups is a result of chance 
rather than a real difference between the groups. The WWC labels a finding statistically significant 
if the likelihood that the difference is due to chance is less than 5% (p < 0.05). 

A substantively important finding is one that has an effect size of 0.25 or greater, regardless 
of statistical significance. 



Please see the WWC Procedures and Standards Handbook (version 2.1) for additional details. 
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